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action by the United States Patent and Trademark Office. The date of abandonment is the day after the expiration 
date of the period set for reply in the office notice or action plus an extensions of time actually obtained. 
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(3) Terminal disclaimer with disclaimer fee - required for all utility and plant applications 
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(4) Statement that the entire delay was unintentional. 
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3. Terminal disclaimer with disclaimer fee 



E Since this utility/plant application was filed on or after June 8, 1995. no terminal disclaimer is required. 



A terminal disclaimer (and disclaimer fee (37 CFR 1 .20(d)) of $ . 
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1 — 1 for other than a small entity) disclaiming the required period of time is enclosed herewith (see 
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abandonment or the delay in filing a petition under 37 CFR 1.137(b) was unintentional (MPEP 711.03(c), 
subsections (lll)(C) and (D)).l 
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numbers (other then a check or credit card authorization form PTO-2038 submitted for payment purposes) is never required by 
the USPTO to support a petition or an application. If this type of personal information Is included In documents submitted to the 
USPTO. petitioners/applicants should consider redacting such personal information from the documents before submitting them 
to the USPTO, Petitioner/applicant is advised that the record of a patent application ia available to the public after publication 
of the application (unless a non-puWication request in compliance with 37 CFR 1 .213(a) te made in the application) or issuance 
of a patent Furthermore, the record from an abandoned application may also be available to the public if the application is 
referenced in a published application or an issued patent (see 37 CFR 1.14). Checks and credit card authorization forms PTO- 
2038 submitted for payment purposes ajejiot retained in the application file and therefore era not publicry available. 
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EXAMINATION FEES 
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3. APPLICATION SIZE FEE ^ , 
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First Named Inventor; 
Application Number: 
Filing Date 
Group Aft Unit 
Examiner Name 



Hermann T"?gf 



Attorney Docket Number; BSSNPQ9 



□ Declaration Submitted with Initial Fifing 
H Declaration Submitted after Initial Filing 
(surcharge (37 CFR 1 .16 (e)) required} 



As a below named inventor. I Hereby declare that: 

My residence, maffing address, and citizenship are as stated below next to my name. 

I believe I em the original, first, and sole inventor (if only one name te listed below) or an original, first, and joint inventor (if 
plural names are listed below) of the subject matter that is claimed and for which a patent is sought on the Invention entitled: 

Pttterflon and Qriooinn of Objects _ _ , • 

the specification of which 



□ is attached hereto. 

OR 

S was filed on 09/16/2003 (MM/DD/YYYY) as U.S. Application Number or PCT International Application 
Number PCT7EP2003/01041 1 and was amended on 03/23/2005 (MM/PP/YYYY) (if applicable). 

I hereby state that I have reviewed and understand the contents of the above-identified specification, including the claims, as 
amended by any amendment specifically referred to above. 

I acknowledge theduty to disclose mformattan that la material to patentability as defined_fcL37 CFR 1.WJndi^hgJpr 
continuation-in-part applications, material information that became available between the Wing date of the pnor application 
and the national or PCT International filing date of the continuation-in-part application. 

I hereby claim foreign priority benefits under 36 U.S.C. 1 19(aHd> or (f). or 365(b) of any foreign appiicallon(s) for patent. 
Inventor's or plant breeder's rights cerUftaete(s)> or 365(a) of any PCT international application that designated at least one 
country other than the United States of America, listed below and have also identified below, by checking the box, any 
foreign application for patent, inventor's or plant breeder's rights certrflcate(s), or any PCT international application having a 
filing date before that of the application on which priority is claimed. 



Prior Foreign Application 
Numbers) 


Foreign Filing Date 
(MM/DWYYYY) 


Country 


Priority 
Not Claimed 


Cei 
A 
Y 


titled Copy 
attached? 
ES NO 


PCT/EP2003/010411 


09/18/2003 


WO 










102 44 2754 


09/23/2002 


DE 










103 38 323.9 


08/21/2003 


DE 

























I hereby appoint Thomas M. Champagne (Reg. No. 36.478) as agent to prosecute the application identified above, and to 
transact all business in the United States Patent and Trademark Office connected therewith. 

Please direct ail correspondence to; 

IP Strategies 

12 'A Wail Street, Suite I 

Ashevilte. North Carolina 28801 

828^53.8600 

828.253.6620 fax 

l hereby dedare tha t si) statementa^mattehe^of-my own knowledge aro trwand that-afhrtetefnef^ 
and belief are believed to be true; and further that these statements were made with the knowledge that wHNT false 
statements and the like so made are punishable by fine or imprisonment, or both, under 16 U.S.C. 1001 and that such willful 
false statements may Jeopardize the validity of the application or any patent issued thereon. 

NAME OF SOLE OR FIRST INVENTOR; Hermann lEfflBL 



Glven Name<9) Family Name or Surname 

U>o - - IfTr — A°\ k f rrf Zoo <r 

Inventor^ Signature 



Residence: 
or MaiRng 
Address 



Btumenstrasee 5 
Street Address 

St Laon-Rot 
City 

66789 



ZiP/Postal Code 



State 
P5 



Country 



Citizenship 



IP Strategies, Asheville, North Carolina 
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WO 2004029864 

Hermann Tropf T50323PC US 

Sensing and Gripping Objects 

The invention relates to sensing, gripping or processing unordered or badly or- 
5 dered or inaccurately positioned parts, particularly, bulk parts, preferably by 
means of a robotic device or other manipulator devices. 

Oh this subject there is a wealth of, mostly academic, literature mainly involved 
with methods of image processing. The methods employed for image analysis are 
10 mostly contour-oriented and model-based. Contour-based methods furnish, as 
compared to surface-based methods, in general, results which are more stable and 
precise and are needed as a rule for partly concealed workpieces. in model-based 
methods the workpiece geometry (or parts thereof) are input to Start with, e r g. 
from CAD data. 

15 

One problem decisive in realizing such a system in practice is that only part of the 
workpiece contours can be reliably represented with known camera and illumina- 
tion arrangements. 

20 Tn DE 3545960 an attempt is made to obtain a plurality of images using just a sin- 
gle camera under various iUumination conditions; although this enhances the 
probability of an edge being represented in one of these images, this is not, how- 
ever, assured, for example, when at the edge the background has the same surface 
properties as the upper workpiece and the same spatial orientation. 

25 

Tt is only in special cases with known illumination arrangements that reliable im- 
age features materialize, for example with bright cylindrical parts resulting in a 
reliably bright reflecting generatrix line. This is why, despite the major iequire- 

1 
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merit in practice, robotic devices have hitherto hardly been put to use for three- 
dimensional bulk gripping applications; achieving this object as applicable in gen- 
eral compatible with actual practice has failed to come about hitherto because 
certain contours under unfavorable conditions are poorly represented or not at all. 
5 Getting round this problem necessitated highly involved mechanical complica- 
tions; for instance, by spreading the parts on a transparent conveyor belt with back 
lighting and - if the parte lie one on the other - with a controllable jolting device 
in the hope of jolting the hems apart so as to represent them fully contoured, all 
this, before they are even localized via an image processing system and gripped 
10 by a robotic device. 

Which contours can be reliably represented and which not, depends on the chance 
spatial orientation of the workpieces with known camera and illumination ar- 
rangements. This makes it extremely difficult to achieve reliable image analysis, 
is especially when the task involved requires a true three-dimensional position de- 
termination of the workpieces (involving in general three position parameters and 
three orientation parameters). 

One way of getting round this, is the transition from contour-oriented processing 
20 to surface three-dimensional analysis by means of structured light. This method is, 
however, technically extremely complicated and necessitates exotic illumination 
components. 

One object of the invention is reliable representation of all, or as many uncon- 
2t> cealed contours of a woikpiece as possible, irrespective of the random spatial ori- 
entation of the workpiece in making use of simple standard illumination compo- 
nents, particularly without stuctured light. 

Another object is to permit learning by simple pointing when setting up the sys- 
30 tern and data comparison in automatic operation without having to input model 
data in advance when setting up the system. 

2 
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These objects are achieved by a method as set forth in claim 1 and claim 12 and 
by a device as set forth in claim ] 8. Advantageous embodiments read from the 
dependent claims. 

In accordance therewith, a method is made available for establishing a data collec- 
tion with the aid of at least one imaging means and at least one illuminating de- 
vice, wherein an object is imaged from at least three different imaging directions 
and illuminated from at least three different illumination directions, each in direct 

10 light, one imaging direction each being substantially opposed to one illumination 
direction, so that from each of the three imaging directions at least one contour 
each of the object appears with a light side and a shadow side of the object, and 
substantially the complete object is imaged from the at least three imaging direc- 
tions by the at least one imaging means, wherein the imaging directions and the 

15 illumination directions, on the one hand, and the object, on the other, are movable 
defined relative to each other with several degrees of freedom, and the images 
and/or the data derived therefrom are stared in the data collection. 

In one aspect of the invention images can be made in various relative positions of 
20 image capture devices and illumination devices, on the one hand, and of the ob- 
ject, on the other. 

In another aspect of the invention changes In the relative positions between the 
images can be captured and assigned to the images and wherein position indica- 
25 tions assigned to the images or to the data derived therefrom are stored in the data 
collection. 

In yet another aspect of the invention several images can be made between which 
the position of the image capture devices and illumination devices is changed mu- 
30 tually. 

3 
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In still another aspect of the invention the position of the image capture devices 
and illumination devices can be determined by a robotic setting. 

5 in another aspect of the invention several images can be made between which the 
position of the object is changed. 

In yet another aspect of the invention the position of the object can be changed by 
means of a robotic device. 

10 

A further aspect of the invention provides a data collection with images and/or 
data derived therefrom, as well as a computer-readable storage medium with data 
which form a data coDection. 

15 In another further a&pect of the invention a method is made available for gripping 
an object from a plurality of objects with the aid of at least one imaging means 
and at least one illuminating device, wherein the object is imaged from at least 
three different imaging directions and illuminated from at least three different 
illumination directions, each in direct light, one imaging direction each being sub- 

20 stamially opposed to one illumination direction, so that from each of the three 
Imaging directions at least one contour each of the object appears with a light side 
and a shadow side of the object and substantially the complete object is imaged 
from the at least three imaging directions by the at least one imaging means. 

2 5 In still another further aspect of the invention reference images and/or data de- 
rived therefrom contained in a data collection as set forth in claim 10 can be used. 

In yet another further aspect of the invention images of the object and images or 
derived data in the data collection can be compared. 

4 
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In yet another further aspect of the invention the illumination devices and the im- 
age capture devices can be arranged substantially star-shaped, the illumination 
directions as viewed from one direction comprising in pairs an angle between 75 
5 degrees and 145 degrees, preferably 120 degrees and as viewed from one direc- 
tion the imaging directions comprising in pairs an angle between 75 degrees and 
145 degrees, preferably 120 degrees. 

In still another further aspect of the invention wherein each imaging can be pcr- 
io formed substantially from the imaging directions via light from the substantially 
opposed illumination direction, preferably by switching and/or by polarization 
and/or spectral filtering and/or by use of at least one color rendering image cap- 
ture device. 

15 A further aspect of the invention provides a computer-readable storage means 
with a method for establishing a data collection. 

Still another further aspect of the invention provides a device for gripping an ob- 
ject from a plurality of objects with at least three image capture devices and at 
20 least three direct illumination devices, wherein one image capture device each is 
substantially opposed to one illumination device, so that from each of the three 
image capture devices at least one contour each of the object can be imaged with a 
light side and a shadow side of the object, and substantially the complete object 
can be imaged by the at least three image capture devices in combination. 

25 

Yet another further aspect of the invention provides a device configured to use 
reference images and/or data derived therefrom contained in a data collection as 
set forth in claim 10. 



5 
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10 



15 



20 



In another aspect of die invention the device may further comprise a substantially 
star-shaped arrangement of illumination devices and image capture devices, 
whereby as viewed from one direction the illumination directions comprise in 
pairs an angle between 75 degrees and 145 degrees preferably 120 degrees and as 
viewed from one direction the imaging directions comprise in pairs an angle be- 
tween 75 degrees and 145 degrees, preferably 120 degrees. 

In another farther aspect of the invention the device may be configured for captur- 
ing images via light from the substantially opposed illumination direction, pref- 
erably by switching means and/or by polarization and/or spectral filters and/or by 
at least one color rendering image capture device. 

The invention will now be detailled by way of the attached drawing in which: 

FIG. 1 is a side view of the camera and illumination arrangement in accordance 
with the present invention; 

FIG. 2 is a view of the arrangement as shown in FIG. 1 as seen vertically from 
above; 

FIG. 3 is a section showing a camera and the illumination thereof in accordance 
with the present invention; and 

FIG. 4 is a workpiece scene and three cameras as well as diagrammatic represen- 
tations of the images captured by the cameras. 

Referring now to FIG. 1 there is illustrated a side view of the camera and illumi- 
nation arrangement with the cameras I, 2, 3 and the illumination devices (shown 
symbolically) 11 (assigned to camera 1), 12 (assigned to camera 2) and 13 (as- 



6 




• SVR:USPTO«ffXRF-1/22 * DWS:2738300 • CSID:7C3 248 9244 • DURATION (mm-ss):09^8 



Sent By: IP Strategies, P.C.; 



703 248 9244; 



Jul-11-06 4:01PM; 



Page 12 



signed to camera 3) as well as the bulk woricpieces 4. The fixed base as shown 
can, of course, be replaced, for example, by a transport conveyer or a container. 

Referring now to FIG. 2 there is illustrated the arrangement as seen vertically 
5 from above. 

Referring now to FIG. 3 there is illustrated sectionwise a camera and the associ- 
ated illumination showing the shadow 5 produced thereby at the edges 6 of the 
workpiece. The opposite edge 7 of the workpiece is evident, as ^seen" by the 

10 camera, only under favorable conditions, for example, against a dark base, 
whereas the edges 6 are reliably „seen" by the camera 1 irrespective of how bright 
the materia] is and the three-dimensional position of the workpiece. Indeed, this 
also applies for bright workpiece surfaces when the illumination is adapted ac- 
cordingly, i.e. the edges 6 produce in the image a contour. Both workpiece sides 

is of the contour are „seen" by the camera, namely on the one hand the workpiece 
surface 9 ilhiminated by the opposite illumination device and, on the other, the 
workpiece surface 9 not illuminated by the opposite illumination device. The 
workpiece surface 9 reflects in any case brightly since it is located approximately 
in the gloss angle (with dull surfaces a small light source is sufficient, but to 

20 achieve this with bright surfaces a larger light source is required), in the direction 
of the camera a shadow 5 exists as „seen" by the camera, i.e. visible (a shadow 
located on the opposite side of the workpiece would e.g. not be visible): it is in 
this that the surface 8 always appears darker than the surface 9, irrespective of the 
surface finish of the workpiece material. This is why the workpiece contour is 

25 always reliably imaged as „seen" by the camera, no matter what the background, 
since the contour is edged by two sides of the same workpiece. 

Although the end of the shadow (shadow edge 10) likewise forms a contour, this 
has the inverse polarity in the image: in the camera image the brightness at the 
30 workpiece contour changes from bright to dark as viewed from top to bottom, at 
the edge of the shadow from dark to bright in thus making it very simple to distin- 

7 
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guish in image analysis workpiece contours from shadow contours (although a 
shadow occuring at the workpiece edge 7 would have inverted polarity, this is 
invisible to the camera). 

5 Referring now to FIG. 4 there is illustrated, as viewed from above, a workpiece 
scene and three cameras together with the three separate images 21 (for camera 1), 
22 (for camera 2), 23 (for camera 3) showing the shadow as a thick edge. 

One particular advantage of the invention is that with the arrangement of at least 
10 three cameras and correspondingly engineered fields of view every contour sec- 
tion in at least One camera is in the situation (edge 6) as shown in FIG. 3 and for 
this camera the contour reliably appears in the image. 

A subsequent image analysis process then has all information as needed to 
is achieve type or position recognition (apart from any geometric ambiguities exist- 
ing basically). All non-concealed contours of the workpiece top surface are relia- 
bly represented in at least one camera. 

These highly advantageous properties in contouring are even maintained in prin- 
20 ciple when separating the channels as it reads from claim 4 or 5 is not performed, 
i.e. when no fully developed shadow forms but just a half-shadow as is the case, 
for instance, when the three illumination devices opposite the cameras are formed 
by a single ring light and no further measures are undertaken for channel separa- 
tion. Although the then resulting half-shadow is not as significant as a full shadow 
25 as is achieved in channel separation, it is sufficient in most cases. 

In three-dimensional analysis the particularly advantage afforded by the described 
arrangement with three cameras located at an angle to each other is that minor 
rotations of the workpiece spatially result in a significant change in a contour im- 
30 age (it being generally the case that several contour images change at the same 

8 
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time) such that three-dimensional position sensing is possible also with good nu- 
merical accuracy and teaching simply by presenting workpieces in various posi- 
tions is possible without having to recourse to model information. 

5 It is particularly of advantage that for three-dimensional position sensing the ge- 
ometry of the workpiece edges does not need to be known when sensing is 
achieved by comparison with the images captured in defined relative positions in 
setting up the system or data derived therefrom in a reference data collection or 
reference data base, since in image analysis workpiece contours are very easy to 

10 distinguish from shadow contours as a function of the ambient conditions (see 
above) without having to make use of prior knowledge as to the workpiece ge- 
ometry. In approaches as known hitherto reliably distinguishing workpiece edges 
from shadow edges first occurred in the scope of the (in generally model-based) 
image analysis process; now making this distinction is done in advance and with- 

15 out prior knowledge, which in turn greatly simplifies subsequent anaylsis in ren- 
dering it more robust. 

Further advantages materialize from moving the cameras and illumination devices 
together against the workpiece. For example, when securing camera and illumina- 

20 don device to the robotic device in setting up the system, precisely the same rela- 
tive image capture and Uhimination situation is simulated as is later with the ro- 
botic device fixed and the part changed in position. Unanticipated effects such as 
e.g. dazzle and shadows occur in precisely the same way during teaching and 
sensing which in turn first makes teaching possible by straight presentation and 

25 comparison in keeping with actual practice. This advantageous situation does not 
exist when cameras and illumination devices are moved separately. In other 
words, simple learning by presentation and sensing by comparison with that pre- 
sented, without model information, is now achieved in conjunction with 

reliable and explicit contour extraction solely at body edges because of 
30 the selected camera and iUumination device arrangement, and 
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precisely reproducible imaging by moving illumination device and 

camera together relative to the workpiece. 
The latter is also achieved when, in addition, also cameras and illumination de- 
vices are mutually movable, but always stand in the same, or at least approxi- 
5 mately the same, relative position to each other when capturing the image. 

This property also results in multiphase functioning being easier and more reliabry 
achievable by the robotic device following a first image capture in a first move- 
ment roughly adap ting firstly the position of the cameras relative to the workpiece 
io and then in subsequent image capture and analysis achieving fine adaptation for 
precise gripping. It is, of course, just as possible that the whole procedure is 
achievable in more than two phases in sequence or continually. 

Although multi-camera arrangements with transmitted, instead of direct, light 
is produce just as reliable contour images, these are achievable more in simple spe- 
cial cases possibly. This applies particularly to multidimensional position sensing; 
in transmitted light arrangements no illuminated surfaces of the parts are to be 
seen, for recognizing jumbled parts, as shown in FIG. I, transmitted light ar- 
rangements are hardly achievable to meet die requirements of actual practice. 

20 

Referring now to FIG. 2 there is illustrated a symmetrical arrangement with 3 
cameras. The arrangement and method as now described relate, of course, just as 
well to arrangements similar in sense but which deviate from a symmetrical ge- 
ometry. Cameras and lamps do not need to be at the same level both one to the 
25 other and mutually, i.e. it is not necessary to divide the 380 degree scope in equal 
angles, there being no basic requirement for the fields of view to cover the 360 
degree scope completely (the singular result of analysis is a function of the work- 
piece geometry, e.g. with symmetries or conversely with highly significant local 
contour shapes). 

30 
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Although FIGs. 1 and 2 show arrangements with cameras directed looking in from 
the outside they must not necessary be directed in this way, e.g. as with work- 
pieces with inner edges such as e.g. circular rings. 

5 Structured illumination is not necessary, although, of course, it is just as possible 
to additionally use structured illumination to achieve e.g. a better separation from 
interference light via subsequent image processing software filters. 

Tt is, of course, just as possible to achieve the described arrangement instead of 
io with three cameras also with an arrangement of mirrors and fewer cameras ar- 
ranged accordingly, and also by means of rigid or flexible light guides. The illu- 
mination devices facing the cameras may comprise, aa already explained by way 
of the example of a ring light, just a single illumination device or ambient illumi- 
nation as existing by chance relative to the cameras. 

15 

The invention relates, of course, not just to robotic devices but also to any ma- 
nipulators with open/closed loop control. 

A further example embodiment of the invention is an arrangement for type and/or 
20 position sensing of unordered or badly ordered, or one or more inaccurately posi- 
tioned parts, particularly, bulk parts, imaged by means of cameras in direct light, 
particularly with the object of gripping or processing, involving 

at least three cameras being directed at the part(s) from different direc- 
tions, 

25 ~ an illumination device being provided for each of these cameras, 

potentially at least one part contour of a common part being in the field 
of view of each of the three cameras, whereby both sides of the com- 
mon part can be seen by the camera, namely, on the one hand, a sur- 
face of the part illuminated by the relative opposite illumination device 
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in each case and, on the other, a surface of the part not illuminated by 
die relative opposite illumination device in each case, 
the illumination devices and the three cameras, on the one hand, and 
the part($), on the other, each being movable relative to the other. 

5 

In this context „potentially u is to be understood that the fields of view of the cam- 
eras are oriented so that a common workpiece can be located in the fields of view 
with one contour each (if this is not the case, e.g. a robotic device with cameras 
assumes a new position to search for a workpicce; when the cameras arc located 
io fixed a transport conveyer, for instance, is advanced until a common workpiece is 
located in the fields of view). 

In another further embodiment of the invention in an arrangement for position 
sensing, cameras and illumination devices are Fitted to a robotic device for defined 
1 5 movement together relative to the part(s). 

in yet another further embodiment thereof in an arrangement for position sensing 
with cameras and illumination devices not fitted to a robotic device, a part can be 
presented, preferably placed, particularly preferably by a robotic device in various 
20 defined positions in the field of view of die cameras when setting up die system. 

In still another further embodiment of the invention the illumination-image cap- 
ture channels can be separated in an arrangement by switching means and/or by 
polarization filters and/or color filters and/or by color cameras. 

25 

In yet another further embodiment of the invention a method for gripping and/or 
processing unordered or badly ordered, or inaccurately positioned parts, particu- 
larly, bulk parts can include an arrangement for position sensing in which image 
capturing by at least three cameras is done separately via the illumination device 
3 o relative to each camera, preferably 

12 
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switching the illumination devices and image capture separate in time, 

and/or by polarized light, and polarization filters at the camera side, 

arid/or by illumination devices having differing spectral ranges, and 
selection of spectral ranges at the camera side preferably by color cam- 
s eras or spectral filters. 

In still another further embodiment of the invention a method for gripping and/or 
processing unordered or badly ordered, or inaccurately positioned parts* particu- 
larly, bulk parts can include a robotic device and an arrangement for position 
io sensing when setting up the system in which images of a sample part are captured 
in various relative positions of cameras and illumination devices, on the one hand, 
and of the sample part, on the other. 

In yet another further embodiment of tins invention in a method for gripping in- 
15 eluding an arrangement fen- position sensing, cameras and illumination devices 
can be moved in common when setting up the system for teaching the parts. 

In still another further embodiment of the invention in a method for gripping in- 
cluding an arrangement for position sensing, a sample part can be presented in 
20 various defined positions in the field of view of the cameras when setting up the 
system for teaching the parts. 
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Claims 



1. 

■ 5 
10 
15 
20 

2. 

25 

3. 

30 



A method for establishing a data collection 

with the aid of at least one imaging means and at least one illuminating 
device, 

wherein an object is imaged from at least three different imaging direc- 
tions and 

illuminated from at least three different illumination directions, each in 
direct light, 

wherein one imaging direction each being substantially opposed to one 
illumination direction, 

so that from each of the three imaging directions at least one contour of 
the object appears with a light side and a shadow side of the object, 
and substantially the complete object is imaged from the at least three 
imaging directions fay the at least one imaging means, 
wherein the imaging directions and the illumination directions, on the 
one hand, and the object, on the other, are movable defined relative to 
each other with several degrees of freedom, 

and wherein the images and/or the data derived therefrom are stored in 
the data collection. 

The method as set forth in claim 1 wherein images are made in various 
relative positions of image capture devices and illumination devices, on 
the one hand, and of the object, on the other 

The method as set forth in claim 2 wherein changes in the relative posi- 
tions between the images are captured and assigned to the images and 
wherein position indications assigned to the images or to the data derived 
therefrom arc stored in the data collection. 
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4. The method as rcl forth in claim 2 or claim 3 wherein the illumination 
devices and the image capture devices are arranged substantially star- 
shaped, the illumination directions as viewed from one direction compris- 
ing in pairs an angle between 75 degrees and 145 degrees, preferably 120 
5 degrees and as viewed from one direction the imaging directions compris- 

ing in pairs an angle between 75 degrees and 145 degrees, preferably 120 
degrees. 

5. The method as set forth in any of the claims 2 to 4 wherein each imaging 
io is performed substantially from the imaging directions via light from the 

substantially opposed illumination direction, preferably by switching 
and/or by polarization and/or spectral filtering and/or by use of at least one 
color rendering image capture device. 

i s 6. The method as set forth in any of the claims 2 to 5 wherein several images. 

are made between which the position of the image capture devices! and il- 
lumination devices is changed mutually, 

7. The method as set forth in claim 6 wherein the position of the image cap- 
20 tare devices and illumination devices is determined by a robotic setting. 

8. The method as set forth in any of the claims 2 to 5 wherein several images 
are made between which the position of the object is changed. 

25 9. The method as set forth in claim 8 wherein the position of the object is 
changed by means of a robotic device. 

10. A data collection with images and/or data derived therefrom, structured as 
set forth in any of the preceding claims, 

30 

11. A computer-readable storage medium with data which form a data collec- 
tion as set forth in claim 10. 



IS 
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12. A method of gripping an object from a plurality of objects 

with the aid of at least one imaging means and at least one illuminating 
device, 

s - wherein the object is imaged from at least three different imaging di- 

rections and 

is illuminated from at least three different illumination directions, each 
in direct light, 

wherein one imaging direction each being substantially opposed to one 
io illumination direction, 

bo that from each of the three imaging directions at least one contour 
each of the object appears with a light side and a shadow side of the 
object, 

and substantially the complete object is imaged from the at least three 
15 imaging directions by the at least one imaging means. 

13 . The method as set forth in claim 12 wherein reference images and/or data 
derived therefrom contained in a data collection as set forth in claim 10 are 
used. 

2D 

14. The method as set forth in claim 12 or 13 wherein images of the object and 
images or derived data in the date collection are compared. 

15. The method as set forth in any of the claims 12 to 14 comprising a sub- 
25 stantiaUy star-shaped arrangement of illumination devices and image cap- 
ture devices, whereby as viewed from one direction the illumination direc- 
tions comprise in pairs an angle between 75 degrees and 145 degrees, 
preferably 120 degrees and as viewed from one direction the imaging di- 
rections comprise in pairs an angle between 75 degrees and 145 degrees, 

3 o preferably 1 20 degrees. 

16. The method as set forth in any of the claims 12 to 15 wherein image cap- 
hiring is performed substantially from the imaging directions via light 
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from the substantially opposed illumination direction, preferably by 
switching and/or by polarization and/or spectral filtering and/or by use of 
at least one color rendering image capture device. 

17. A computer-readable storage means comprising & program code which 
implements the method as set form in any of the claims 12 to 16 when 
loaded in a computer. 

18. A device for gripping an object from a plurality of objects 



10 



with at least three image capture devices and 

at least three direct illumination devices, wherein one image capture 
device each is substantially opposed to one illumination device, 
so that from each of the three image capture devices at least one con- 
as tour each of the object can be imaged with a light side and a shadow 
side of the object, 

and substantially the complete object can be imaged by the at least 
three image capture devices in combination. 

ao 19, The device as set forth in claim 18 configured to use reference images 
and/or data derived therefrom contained in a data collection as set forth in 
claim 10. 

20. The device as set forth in claim 18 or 19 further comprising a substantially 
25 star-shaped arrangement of illumination devices and image capture de- 
vices, whereby as viewed from one direction the illumination directions 
comprise in pairs an angle between 75 degrees and 145. degrees, preferably 
120 degrees and as viewed from one direction the imaging directions 
comprise in pairs an angle between 75 degrees and 145 degrees, preferably 

30 120 degrees. 

21. The device as set form in any of the claims 18 to 20 configured for captur- 
ing images via light from the substantially opposed illumination direction, 
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preferably by switching means and/or by polarization and/or spectral fil- 
ters and/or by at least one color rendering image capture device. 
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- 1 - 

Preliminary amendment 

PCT/BP2003/0104U 

BSSNP09 

A mendments to. the claims 

5 Please amend the claims as follows: 

1 . (original) A method for establishing a data col lection 

with the aid of at least one imaging means and at Least one illuminating 
device, 

10 - wherein an object is imaged from at least three different imaging direc- 
tions and 

illuminated from at least three different illumination directions, each in 
direct light, 

wherein one imaging direction each being substantially opposed to one 

is illumination direction, 

so that from each of the three imaging directions at least one contour of 
the object appears with a light side and a shadow side of the object 
and substantially the complete object is imaged from the at least three 
imaging directions by the at least one imaging means, 

20 - wherein the imaging directions and the illumination directions* on the 
one hand, and the object, on the other, are movable defined relative to 
each other with several degrees of freedom, 

and wherein the images and/or the data derived therefrom are stored in 
the data collection. 

25 

2. (original) The method as set forth in claim 1 wherein images are made in 
various relative positions of image capture devices and illumination de- 
vices* on the one hand, and of the object, on the other. 
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2 



(original) The method us set forlh in claim 2 wherein changes in the rela- 
tive positions between the images are captured and assigned to the images 
and wherein position indications assigned to the images or to the data de- 
rived therefrom are stored in the data collection. 



I. (currently amended) The method as set forth in claim 2 ^herein the illu- 
mination devices and the image capture devices are arranged substantially 
star-shaped, the illumination directions as viewed from one direction com- 
10 prising in pairs an angle between 75 degrees and 145 degrees, preferably 

120 degrees and as viewed from one direction the imaging directions 
comprising in pairs an angle between 75 degrees and 145 degrees, prefera- 
bly 120 degrees. 



Deleted; or dnm 3 



13 



20 



(currently amended) The method as set forth in £lajm 2_ wherejneach im- , 
aging is performed substantially from the imaging directions via light from 
the substantially opposed illumination direction, preferably by switching 
and/or by polarization and/or spectral filtering and/or by use of at least one 
color rendering image capture device. 

(currently amended) The method as set forth in f laim 2 wherein several , 
images are made between which the position of the image capture devices 
and illumination devices is changed mutually. 



any of the claim* 2 to 4 



: any of chc daima 2 to 5 



25 7. (original) The method as set forth in claim 6 wherein the position of the 
image capture devices and illumination devices is determined by a robotic 
setting. 
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DetateO: any of the dwn* 2 to S 



(currently amended) The method as set forth in fjaim 2 w herein several ... 
images are made between which the position of the object is changed. 



9. (original) The method as set forth in claim 8 wherein the position of the 
5 object is changed by means of a robotic device. 

10. (currently amended) A data collection with images and/or data derived 

4 Deleted: **y of the gwjjng china J 

therefrom, structured as set forth Ir^clajrrM . _ • 

10 11. (original) A computer-readable storage medium with data which form a 
data collection as set forth in claim 10. 



12. (original) A method of gripping an object from a plurality of objects 

1 5 - with the aid of at least one imaging means and at least one iUuminating 
device, 

wherein the object is imaged from at least three different imaging di- 
rections and 

is illuminated from at least three different illumination directions, each 
20 in direct light, 

wherein one imaging direction each being substantially opposed to one 
illumination direction, 

so that from each of the three imaging directions at least one contour 
each of the object appears with a light side and a shadow side of the 
25 object, 

and substantially the complete object is imaged from ihe at least three 
imaging directions by the at least one imaging means. 
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LL (currently amended) ^method ftf gripping an object from a plurality of* < j ***** The , ) 

"*^ s: I Formatted: Buitetg aad Numbering ) 
objects • r^^x 



with the aid of at least one imaging means and a t least one illuminating 
device. 

wherein the object is imaged from at least three different imaging di- 
rections and 

is illuminated from at least three different iHummatio n di rections, each 
indirect light. 

wherein one imagi ng direction each being substantially opposed to one 
10 illumination direction. 

SO that from each of the three imaging directions at least one contour 
each of the object appears with a light side and a shad ow side of the 
object. 

and substantially the complete object is imaged from the at least three 
15 imaging directions bv the a t least one imaging means. 



**8<*ft>nhin tiairo 12 ] 



wherein reference images and/or data derived therefrom c ontained In a data" * ^ *>rmatfrd: to dgfc ugt: 02T J 
collection, as set forth in claim 10 are used. 



| 14. (currently amended) The method as set forth in claim 12, wherein images_. ) 

20 of the object and images or derived data in the data collection are com- 

pared. 

| 1 5. (currently amended) The method as set forth m£lairoLl2.«)mprisina. a sub- . - HSS^S.^^IS'Jii", J 

stantlally star-shaped arrangement of Illumination devices and image cap- 
is ture devices, whereby as viewed from one direction the illumination direc- 
tions comprise in pairs an angle between 75 degrees and 145 degrees, 
preferably 120 degrees and as viewed from one direction the imaging di- 
rections comprise in pairs an angle between 75 degrees and 145 degrees, 
preferably 120 degrees. 

30 
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(~De*eted: my of the ) 

(currently amended) The mclhod as set forth in gclaim 12 wherein iinafte - J^^^^u^^ j 

capturing is performed substantially from the imaging directions via light 
from the substantially opposed illumination direction, preferably by 
switching and/or by polarization and/or spectral filtering and/or by use of 
at least one color rendering image capture device. 



1Q 



17, (currently amended) A computer-readable storage means comprising a 
program code which implements the method as set forth in T claim 12 w hen 
Loaded in a computer. 

18. (original) A device for gripping an object from a plurality of objects 



- fpelcited; «ny o r ifcc deans 17 tu 16 



15 



20 



25 



with at least three image capture devices and 

at least three direct illumination devices, wherein one image capture 
device each is substantially opposed to one illumination device, 
so that from each of the three image capture devices at least one con* 
tour each of the object can be imaged with a light side and a shadow 
side of the object, 

and substantially the complete object can be imaged by the at least 
three image capture devices In combination. 

19. (currently amended) A^ytee /or gripgin^an object from a plurality of - 



with at least three image capture devices and 

at least three direct illumination devices^ wh ^lD ^ capture 
device each is substantial tv opposed to one illumination device , 
so that from each of the three image capture devices at least one con-"' 
tour each of the object can be imaged w ith a light side and a shadow 
sideoftheobiecL 
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z and substantially the complete object can be imaged bv the at least 

three image capture device? in, wmfrinqflpp, 
which is configured to use reference images mid/or data deriv ed therefrom con- 
tained in a data collection as set forth in claim 10. 

5 

I 20. (currently amended) The device as set forth in claim 1 8 .Aii^herj^ • {° e>e * <>Sor ! 2 _J 



10 



a substantially star-shaped arrangement of illumination devices and image 
capture devices, whereby as viewed from one direction the illumination di- 
rections comprise in pairs an angle between 75 degrees and 145 degrees, 
preferably 120 degrees and as viewed from one direction the imaging di- 
rections comprise in pairs an angle between 75 degrees and 145 degrees, 
preferably 120 degrees. 



21. (currently amended) The device as set forth in starm J 8 configured for 
capturing images via light from the substantially opposed illumination di- 
rection, preferably by switching means and/or by polarization and/or spec- 
tral filters and/or by at least one color rendering image capture device. 




any of tfac claims 18 to 20 ) 



20 
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Preliminary amendment 

PCT/EP2003/010411 

BSSNP09 

Amendments to the claims 

5 Please amend the claims as follows: 

1 . (original ) A method for establishing a data collection 

with the aid of at least one imaging means and at least one illuminating 
device, 

10 - wherein an object is imaged from at least three different imaging direc- 
tions and 

illuminated from at least three different illumination directions, each in 
direct light, 

wherein one imaging direction each being substantially opposed to one 

l s illumination direction, 

so that from each of the three imaging directions at least one contour of 
the object appears with a light side and a shadow side of the object, 
and substantially the complete object is imaged from the at least three 
imaging directions by the at least one imaging means, 

20 - wherein the imaging directions and the illumination directions, on the 
one hand, and the object, on the other, are movable defined relative to 
each other with several degrees of freedom, 

and wherein (he images and/or the data derived therefrom are stored in 
the data collection. 

25 

2. (original) The method as set forth in claim 1 wherein images are made in 
various relative positions of image capture devices and illumination de- 
vices, on the one hand, and of the object, on the other. 
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3. (original) The method as set forth in claim 2 wherein changes in the rela- 
tive positions between the images are captured and assigned to the images 
and wherein position indications assigned to the images or to the data dc- 

5 rived therefrom are stored in the data collection. 

4. (cuirently amended) The method as set forth in claim 2 or claim4 wherein 
the illumination devices and the image capture devices are arranged sub- 
stantially star-shaped, the illumination directions as viewed from one di- 

tO rection comprising in pairs an angle between 75 degrees and 145 degrees, 

preferably 120 degrees and as viewed from one direction the imaging di- 
rections comprising in pairs an angle between 75 degrees and 145 degrees, 
preferably 120 degrees. 

15 5. (currently amended) The method as set forth in any of th e claims 2 to 4 
claim 2 wherein each imaging is performed substantially from the imaging 
directions via light from the substantially opposed illumination direction, 
preferably by switching and/or by polarization and/or spectral filtering 
and/or by use of at least one color rendering image capture device. 

20 

6. (currently amended) The method as set forth in any of the claims 2 to 5 
claim 2 wherein several images are made between which the position of 
the image capture devices and illumination devices is changed mutually. 

25 7. (original) The method as set forth in claim 6 wherein the position of the 
image capture devices and illumination devices is determined by a robotic 
setting. 
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5 9. 
10. 

10 

11. 
12. 

15 



20 



(currently amended) The method as set forth in any of the olaimu 2 to 5 
claim 2 wherein several images are made between which the position of 
the object is changed. 

(original) The method as set forth in claim 8 wherein the position of the 
object is changed by means of a robotic device. 

(currently amended) A data collection with images and/or data derived 
therefrom, structured as set forth i n ony of tiro pn- f f riin c rl ni i m f teim 1 . 

(original) A computer-readable storage medium with data which form a 
data collection as set forth in claim ) 0. 

(original) A method of gripping an object from a plurality of objects 

with the aid of at least one imaging means and at least one illuminating 
device, 

wherein the object is imaged from at least three different imaging di- 
rections and 

is illuminated from at least three different illumination directions, each 
in direct light, 

wherein one imaging direction each being substantially opposed to one 
illumination direction, 

so that from each of the three imaging directions at least one contour 
each of the object appears with a light side and a .shadow side of the 
object, 

and substantially the complete object is imaged from the at least three 
imaging directions by the at least one imaging means. 
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13. ( currently amended) The-Ajnethod as s e t forth in claim ±2 -gf gripping an 
object from a plurality of objects 

z with the aid of at lea^t ™»p ima ging means and at least one illuminating 

device. 

5 where in the object is imaged from at least three different imaging di- 

rections and 

- . . is iU uminated from at least three different illumination directions, each 
indirect light, 

wherein one imaging direction each being substantially opposed to one 
10 iUuminatton direction, 

- so that from each of the three imaging directions at least one contour 

each of the object appears with a light side and a shadow side of the 
pfyjqcfc 

and substantially the complete object is imaged from the at least three 
15 imaging directions bv the at least one imaging means. 

wherein reference images and/or data derived therefrom contained in a data 
collection as set forth m claim 10 are used. 



14. (currently amended) The method as set forth in claim 12 e*-43~wherein 
20 images of the object and images or derived data in the data collection are 

compared. 

15. (currently amended) The method as set forth in any of th fr olaimo 12 to 14 
claim 12 c omprising a substantially star-shaped arrangement of iUumina- 

25 tion devices and image capture devices, whereby as viewed from one di- 

rection the iUumination directions comprise in pairs an angle between 75 
degrees and 145 degrees, preferably 120 degrees and as viewed from one 
direction the imaging directions comprise in pairs an angle between 75 de- 
grees and 145 degrees, preferably 120 degrees. 
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16. (currently amended) The method as set forth in any of th e-c laims 12 to 15 
claim 12 w herein image capturing is performed substantially from the im- 
aging directions via light from the substantially opposed illumination di- 
rection, preferably by switching and/or by polarization and/or spectral fil- 

5 tering and/or by use of at least one color rendering image capture device 

17. (currently amended) A computer-readable storage means comprising a 
program code which implements the method as set forth in any of th e 
Almm*U24e»46 - claim 12 when loaded in a computer. 

10 

1 8. (original) A device for gripping an object from a plurality of objects 

with at least three image capture devices and 

at least three direct iUuminatioo devices, wherein one image capture 
l5 device each is substantially opposed to one illurnination device, 

so that from each of the three image capture devices at least one con- 
tour each of the object can be imaged with a light side and a shadow 
side of the object, 

and substantially the complete object can be imaged by the at least 
20 three image capture devices in combination. 



(currently amended) ?fee-Adevicc as-set forth in claim 18 for pipping m 
A bject from a plurality of objects 

with at least three imag^ ^pture devices and 

at least three direct illumination devices, wh erein one image capture 
rtevi ^ each js substanti ally opposed to one illumination, (fcvice, 
so that from each of the three image c apture devices at least one cc-n- 
r ^r p^H of the object ca n be imaged with a light side and a shadow 
qtdft of the object. 
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and substantially the complete object can be imaged by the at 

three image capture devices in combination, 
which is configured to use reference image s and/or data derived therefrom con- 
tained in a data collection as set forth in clairn 10. 

5 

20. (currently amended) The device as set forth in claim 18 e^44Murther com- 
prising a substantially star-shaped arrangement of illumination devices and 
image capture devices, whereby as viewed from one direction the illumi- 
nation directions comprise in pairs an angle between 75 degrees and 145 

10 degrees, preferably 120 degrees and as viewed from one direction the im- 

aging directions comprise in pairs an angle between 75 degrees and 145 
degrees, preferably 120 degrees. 

21 . (currently amended) The device as set forth in any of the olaimo 18 to 20 
15 claim 18 c onfigured for capturing images via light from the substantially 

opposed illumination direction, preferably by switching means and/or by 
polarization and/or spectral filters and/or by at least one color rendering 
image capture device. 
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